Improvement Opportunities and Suggestions for Benchmarking

During the past 10 years, the amount of effort put on setting up benchmarking repositories has considerably increased at the organizational, national and even at international levels to help software managers to determine the performance of software activities and to make better software estimates. This has enabled a number of studies with an emphasis on the relationship between software product size, effort and cost drivers in order to either measure the average performance for similar software projects or to develop estimation models and then refine them using the collected data. However, despite these efforts, none of those methods are yet deemed to be universally applicable and there is still no agreement on which cost drivers are significant in the estimation process. This study discusses some of the possible reasons why in software engineering, practitioners and researchers have not yet been able to come up with reasonable and well quantified relationships between effort and cost drivers although considerable amounts of data on software projects have been collected. An improved classification of application types in benchmarking repositories is also proposed.